REMARKS 

Applicants request favorable reconsideration and allowance of the subject application in 
view of the preceding amendments and the following remarks. 

Claims 1-70, 92-97 and 100-150 having been cancelled, Claims 71-91, 98, 99 and 151- 
165 are presented for examinataion. Claims 71, 80, 98, 99, 151, 157 and 163 are independent. 
Claim 99 has been amended to clarify features of the subject invention. Support for these 
changes can be found in the original application, as filed. Therefore, no new matter has been 
added. 

Applicants request favorable reconsideration and withdrawal of the rejections set forth in 
the above noted Office Action. 

Claim 99 has been amended to correct a typographical error. 

Claims 71 75, 78 84, 87 91, 98 100 and 151 165 have been rejected under 35 USC 103(a) 
as being unpatentable over US patent 5,729,471 (Jain et al.) in view of US patent 6,360,1 16 
(Jackson, Jr. et al.). Claims 76, 77, 85 and 86 were rejected under 35 USC 103(a) as being 
unpatentable over US patent 6,222,551 (Schneider), these rejections are respectfully traversed. 

Pending independent Claims 71 and 98 are directed to a method of generating model data 
representative of a a three dimensional model of an object from input signals representative of 
camera images of the object taken from plural camera positions. According to the method, a set 
of icons each associated with a respective one of camera images of the object are displayed. A 
selection signal is received responsive to user actuation of an input unit whereby the selection 
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signal identifies a selected icon. A selected camera image is determined from the set of camera 
images corresponding to the selected icon. The selected image is displayed and position data 
representative of a selected camera position from which the selected image was taken is 
determined. A model image representative of a view of the model is generated in accordance 
with the model from a viewpoint corresponding to the position data. The model image is 
displayed for visual comparison with the selected image by the user. 

Pending independent Claim 151 is directed to a method of processing data defining a 
three dimensional computer mode of an object, the including data defining plural camera images 
of the object and data defining the viewpoint of each camera image relative to the computer 
model. According to the method, a set of icons each associated with a respective one of camera 
images of the object are displayed. A selection signal is received responsive to user actuation of 
an input unit whereby the selection signal identifies a selected icon. The camera image 
associated with the selected icon and the viewpoint of the identified camera image are identified. 
The image data of the identified camera image is displayed and the three dimensional computer 
model data is rendered to generate image data showing a virtual image of the three dimensional 
computer model from the viewpoint of the identified camera image. The virtual image is 
displayed for visual comparison with the camera image. 

Pending Claims 80 and 99 are directed to apparatus that generates model data 
representative of a three dimensional model of an object from input signals representative of a set 
of camera images of the object taken from plural camera positions. In the apparatus, a display 
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unit displays a set of icons each associated with a respective one of the camera images of the 
object. A receiving unit receives a selection signal in response to user actuation of an input unit. 
The selection signal identifies a selected one of the icons. A determining unit determines a 
selected camera image from the set of camera images corresponding to the selected icon. The 
display unit operates to display the selected image. A determining unit determines position data 
representative of a selected camera position from which the selected image was taken and a 
generating unit generates, in accordance with the model, a model image representative of a view 
of the model from a viewpoint corresponding to the position data. A control unit controls the 
display unit to display the model image for visual comparison with the image selected by the 
user. 

Pending Independent Claims 157 and 163 are directed to apparatus that processes data 
defining a three dimensional computer model of an object. The data including data defining 
plural camera images of the object and data defining the viewpoint of each camera image relative 
to the computer model. In the apparatus, an icon data generator operates to generate icon data 
defining plural icons for display. Each icon is associated with a respective one of the camera 
images of the object. A selection signal receiver operates to receive a selection signal generated 
in response to user actuation of an input device defining a user icon selection. A camera data 
identifier operates to identify the camera image associated with the selection icon and the 
viewpoint of the identified camera image. A renderer operates to render the three dimensional 
computer model data to generate image data defining a virtual image of the three dimensional 
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computer model from the viewpoint of the identified camera image and a display data generator 
operates to generate image data for display that displays the virtual image and the identified 
camera image for visual comparison. 

In Applicants' view, Jain et al. discloses Machine dynamic selection of one video 
camera/image of a scene from multiple video cameras/images of the scene in accordance with a 
particular perspective on the scene, an object in the scene, or an event in the scene in which video 
images of the scene are selected, or even synthesized, in response to a viewer-selected (i) spatial 
perspective on the scene, (ii) static or dynamic object appearing in the scene, or (iii) event 
depicted in the scene. Multiple video cameras, each at a different spatial location, produce 
multiple two-dimensional video images of the real-world scene, each at a different spatial 
perspective. Objects of interest in the scene are identified and classified by computer in these 
two-dimensional images. The two-dimensional images of the scene, and accompanying 
information, are then combined in the computer into a three-dimensional video database, or 
model, of the scene. The computer also receives a user/viewer-specified criterion relative to 
which criterion the user/viewer wishes to view the scene. From the (i) model and (ii) the 
criterion, the computer produces a particular two-dimensional image of the scene that is in "best" 
accordance with the user/viewer-specified criterion. This particular two-dimensional image of 
the scene is then displayed on a video display. From its knowledge of the scene and of the objects 
and the events therein, the computer may also answer user/viewer-posed questions regarding the 
scene and its objects and events. 



19 



In Applicants* opinion, Jackson, Jr. et al. discloses a brachytherapy system for prostate 
cancer treatment with computer implemented systems and processes to facilitate pre-operative 
planning and post-operative evaluations that assist a user in preparing a brachytherapy pre- 
operative plan and post-operative evaluation for prostate cancer. Image scans are loaded into the 
system. A user places a matrix on the image scans and outlines the contours the anatomical 
structures in the image scans. The user then places seeds on the various image scans in a well 
known manner to treat the prostate cancer. The system provides for real-time isodose 
calculations as the user edits the pre-operative plan. The user may also view a three-dimensional 
view of the isodose levels on the anatomical structure, as well as the surface of a given isodose 
level. 

According to the invention of Claims 71, 80, 98, 99, 151, 157 and 163, a set of icons are 
displayed with each icon associated with a respective one of camera images of the object, a 
selection signal received responsive to user actuation of an input identifies a selected icon and a 
selected camera image of the camera images corresponding to the selected icon is determined. 
Position data representative of the selected camera position is determined and a model image 
representative of th model from a viewpoint of the positions data is generated. The model image 
is displayed for visual comparison with the image selected by the user. Advantageously, a user 
is allowed to compare the three dimensional computer model by way of a rendered image thereof 
with a camera image. The user can then determine whether any features on the object are shown 
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in the camera image that are not present in the three dimensional model and the three 
dimensional computer model can be refined as, for example, disclosed at lines 8-21 of page 8. 

Jain et al. is directed to a system that allows a view to select video images to be displayed 
by selecting a desired (i) spatial perspective on the scene, (ii) a static or dynamically moving 
object appearing in the scene, or (iii) an event depicted in the scene (see column 7 lines 20 24 
and column 16 lines 10 28). Multiple video cameras, each at a different spatial location, produce 
multiple two-dimensional images of the real-world scene, each at a different spatial perspective. 
Objects of interest in the scene are identified and classified in these two-dimensional images. 
These multiple two-dimensional images of the scene and their accompanying object information 
are then combined in a computer into a three-dimensional video database, or model, of the scene 
(see column 8 lines 26 34). Images satisfying the user's request are then generated and displayed. 
These may be virtual images or camera images, depending upon the system (see column 18 lines 
24 29). 

In Jain et al, a viewer wishing to view a scene from specific perspective may specify a 
real camera or only the desired general location of a camera without knowing if any camera is in 
such a location (see lines 10-16 of column 16). In order to specify a camera, the user must select 
a camera from a list (see line 7 of column 21). To enable the system to determine which images 
meet the user's object or event selection criteria and to be displayed, a multi-perspective 
perception system is provided to permit live recognition and coverage of objects and events in an 
extended environment (see column 25 lines 44 48). Section 9 of Jain and Figures 19a 19d relate 



21 



only to the multi-perspective perception system independent of the Multiple Perspective 
Interactive video system of which it is a part (see column 29 lines 22 31). Figures 19a 19d only 
illustrate the results of object detection and tracking by the system and are directed away from the 
user interface of the MPI video system described in previous sections of Jain, and have nothing 
to do with any image-selection user interface. As a result, Jain et al. does not teach or suggest 
allowing a user to compare a three dimensional computer model (i.e., a rendered "model" 
image") with a camera image so that the user can determine whether any features on the object 
are shown in the camera image but are not present in the three-dimensional computer model and 
thereby determine whether the three-dimensional model requires refinement to include the 
features as in the present invention. Accordingly, Jain et al. which only discloses displaying an 
image of a scene in accordance with user selection criteria fails to teach or suggest the features of 
Claims 71, 80, 98, 99, 151, 157 and 163. 

The Examiner has cited Figs. 19a 19d of Jain et al. as disclosing the feature of displaying 
icons each associated with a respective camera image, and column 16 lines 10 16 as disclosing 
the feature of receiving a selection signal responsive to user actuation of an input means whereby 
the selection signal identifies a selected camera image. It is a feature of Claims 71, 80, 98, 99, 
151, 157 and 163 that a selection signal identifies a selected one of the icons but does not recite a 
selection signal that identifies a selected camera image as in Jain et al. Further in Jain et al.'s 
system, the selection of a camera by a user is not performed in response to a display of images. 
Rather, section 9 which refers to Figs. 19a and 19b relates only to the multi-perspective 
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perception system independent of the Multiple Perspective Interactive (MPI) of which it is a part 
(see column 29, lines 22-31). Figs 19a-19e form no part of any user interface system for 
selecting images. Additionally, column 16, lines 10-16 cited by the Examiner is devoid of about 
how the user selects a camera. In particular, Jain et al specifically teaches away from a selection 
signal identifying a selected icon in teaching that a viewer must choose a camera using a camera 
list in the user interface of the MPI video system. 

Column 9, lines 25-42 and column 16, lines 10-16 of Jain et al. have been cited by the 
Examiner as disclosing the feature of determining a selected camera image from the set of 
camera images corresponding to the user selection. As aforementioned, Jain et al. specifically 
teaches that a user directly selects a camera from a list (see column 21, line 7). Jain is devoid of 
suggesting any processing to determine a selected camera image from the set of camera images 
corresponding to an icon selected by a user. Column 9, lines 25-42, and column 16, lines 10-16 
of Jain et al., merely teach that a user selects a required perspective but fails to suggest anything 
as to how this is performed (see column 21, line 7, which explains that this is done by directly 
selecting a camera from a list). 

In regard to the citing of column 9, lines 25-42 and column 16, lines 10-16 of Jain et al. 
for the feature of determining a selected camera image from the set of camera images 
corresponding to the user selection, Jain specifically requires that a user directly select a camera 
from a list (see column 21, line 7) and is silent as to any processing to determine a selected 
camera image from the set of camera images corresponding to an icon selected by a user. 
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Figs. 19a-19d have been cited as displaying selected images. As clearly disclosed in Jain et al., 
the images of Figs. 19a-19d show all of the available camera images and there is no suggestion of 
the feature of Claims 71, 80, 98, 99, 151, 157 and 163 of selecting one of the images or 
displaying a selected image. The images displayed in Figs. 19a-19d cannot be selected by a user 
and continue to be displayed independent of user action. Figs. 19a-19d only show the results of 
image detection and tracking and do not form any part of an image selection user interface. Jain 
et al. may also teach scene analysis by extracting three dimensional data for all images in order to 
perform camera calibration and to determine the positions of players as disclosed from column 
21, line 63 to column 22, line 7. The process performed in Jain et al, however, is not performed 
in response to any image selection and fails in any manner to teach of suggest the feature of the 
present invention of determining position data represented of a selected camera position from 
which a selected image was taken. 

It is a further feature of the claimed invention that a model image representative of a view 
of the model from a viewpoint correspond to the position data representative of a selected camera 
position is generated in accordance with the model. Jain et al., at column 34, lines 25-29, fails in 
any manner to teach how the images shown in Figs. 19a-19d are generated. As aforementioned, 
the images of Figs. 19a-19d show all of the available camera images, cannot be selected by a user 
and continue to be displayed independent of user action. Figs. 19a-19d only show the results of 
image detection and tracking and do not form any part of an image selection user interface. As a 
result, it is not seen that the images shown in Figs 19a-19d suggest the features of generating in 
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accordance with the model a model image representative of a view of the model from a 
viewpoint corresponding to the position data and displaying the model image for comparison 
with the selected image by the user. 

Jackson, Jr. et al. may teach a system for assisting a user in preparing a brachytherapy 
pre-operative plan and post-operative evaluation for prostate cancer in which a particular image 
may be selected from a number of thumbnail windows. Fig. 2 of Jackson, Jr. et al. shows a 
number of thumbnail windows (21) from scanned images. A particular image may be selected by 
activating a thumbnail 21 to cause the image to be displayed in a working window 20 (see column 
7, lines 17-33). Jackson, Jr. et al, however, only teaches that the selection of a thumbnail of a 
scan image merely results in the same scan image being displayed in larger format within 
working window 20 but fails in any manner to suggest anything about selecting a camera image 
from a set of camera images corresponding to a selected icon. It is not seen that the addition of 
selecting a scanned image to be displayed in an enlarged format is in any manner related to Jain 
et al. f s system of multiple video cameras, each at a different spatial location, that produces 
multiple two-dimensional images of the real-world scene, each at a different spatial perspective. 

If, arguendo, the Jackson, Jr. et al.'s brachytherapy arrangement with enlargement of a 
selected thumbnail image were combinable with Jain et al., we note that the cited combination 
only displays a camera image in larger format in response to the selection of the camera image 
but fails in any manner to suggest the features of Claims 71, 80, 98, 99, 151, 157 and 163 of 
generating a model image representative of a view of the model from a viewpoint corresponding 
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to the camera position from which the selected image was taken and displaying the model image 
for visual comparison with the selected image by the user. Accordingly, it is believed that 
Claims 71, 80, 98, 99, 151, 157 and 163 are completely distinguished from any combination of 
Jain et al. and Jackson, Jr. et al. and are allowable. 

For the foregoing reasons, Applicants submit that the present invention, as recited in 
independent claims 71, 80, 98, 99, 141, 157 and 163, is patentably defined over the cited art, 
whether that art is taken individually or in combination. 

Dependent claims 72-79, 81-90, 152-156, 158-162, 164 and 165 also should be deemed 
allowable, in their own right, for defining other patentable features of the present invention in 
addition to those recited in their respective independent claims. Further individual consideration 
of these dependent claims is requested. 

Applicants further submit that the instant application is in condition for allowance. 
Favorable reconsideration, withdrawal of the rejections set forth in the above-noted Office 
Action and an early Notice of Allowance are requested. 
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Applicants' attorney, David A. Divine, maybe reached in our Washington, D.C. office by 
telephone at (202) 530-1010. All correspondence should be directed to our address listed below. 



Respectfully submitted, 

Jack S. ffubert 
Attorney for Applicants 
Registration No. 24,245 

FITZPATRICK, CELLA, HARPER & SCINTO 
30 Rockefeller Plaza 
New York, New York 101 12-3800 
Facsimile: (212)218-2200 

JSC/lip 

DC_MAIN201810v1 
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